Skip to content

fix(sanitization): secret exposure in function and agent trace spans - #6000

Open
BillLeoutsakosvl346 wants to merge 9 commits into
stagingfrom
fix/secrets-sanitization-trace-spans
Open

fix(sanitization): secret exposure in function and agent trace spans#6000
BillLeoutsakosvl346 wants to merge 9 commits into
stagingfrom
fix/secrets-sanitization-trace-spans

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Added an environment-aware sanitizer that restores configured {{ENV_VAR}} references in trace/display values, including nested tool calls, timing segments, errors, object keys, and URL-encoded values.
  • Preserved unresolved environment references in Function display/error source while continuing to resolve plaintext runtime code.
  • Applied sanitization at the BlockExecutor observability boundary without changing handler inputs, runtime state, handled errors, retries, or resume execution.
  • Registered the sanitizer at the workflow logging boundary so Overview final output, synthetic errors, fallbacks, final spans, and telemetry copies are also sanitized.
  • Prevented streaming completion and human-in-the-loop resume paths from reintroducing raw values into already-sanitized logs.
  • Kept raw outputs available for API responses, downstream execution, retry/resume state, and billing calculations.

Root cause

Block inputs and provider/tool outputs were logged after environment references had been resolved. Function display code also resolved environment values, and post-executor paths could overwrite sanitized logs with raw streaming, pause-state, or workflow final-output data immediately before persistence.

Impact

New trace spans and Overview workflow output show configured references such as {{AWS_SECRET_ACCESS_KEY}} instead of plaintext values. Runtime tool/function execution, API responses, downstream block behavior, billing, retries, pause/resume, and persisted runtime state remain unchanged. Existing historical logs are not rewritten.

Validation

  • Independent senior review with a second review after addressing all findings
  • 216 targeted Vitest tests across sanitizer, resolver, BlockExecutor, execution core, logging completion, trace spans, Function handler, snapshots, streaming, and HITL resume
  • App TypeScript check
  • Targeted Biome lint/format on all changed files
  • API validation contract audit
  • git diff --check

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 1, 2026 3:14am

Request Review

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor

cursor Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Changes span secret provenance, internal-auth-gated response fields, and every path that persists or streams execution observability; mistakes could leak plaintext or break retry/resume/logging completion semantics.

Overview
This PR extends the sanitization work so observability surfaces (workflow SSE, terminal console, log export/detail, Overview materialization) show configured {{ENV_VAR}} references instead of resolved plaintext, while runtime execution, API outputs, billing, retry/resume state, and handler inputs stay raw.

Private provenance transport: Internal JWT callers can opt in via x-sim-request-private-tool-metadata to receive __resolvedSecretNames (Function {{}} substitutions, scoped to env vs params and secret mount scope) or encrypted __resolvedSecretTraceProvenance on Function, MCP tool execute, Mothership execute, and workflow execute responses. Session and external API callers never get these fields.

Execution route behavior: Workflow execute adds session-only inputFromExecutionId, prefers server-stored snapshots for run-from-block when executionId is present, and attaches display projections on block complete/error, stream chunks, and terminal errors through LoggingSession. Legacy workflow log completion restores trusted provenance from stored execution state before projecting traces.

Client/UI: The builder terminal and execution utils consume server display fields (and clear live thinking when projection is structural-only); final finalBlockLogs reconciliation replaces earlier live projections. Logs retry now reuses execution by ID instead of pulling input from detail payloads.

Other: Functional block outputs for execution status use a shared collector with 409 when unavailable; background workflow/resume jobs distinguish core-finalized errors from faults that still need safeCompleteWithError.

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stops resolved environment secrets from appearing in workflow trace spans and Overview logs while keeping runtime execution and API payloads unchanged.

  • Adds ResolvedSecretTraceRegistry plus Aho–Corasick-style projection (trace-secret-projection) to restore {{ENV_VAR}} (or anonymous redaction) in display traces.
  • Activates secrets only on successful {{NAME}} resolution; carries encrypted provenance across internal boundaries (MCP, function execute, mothership, nested workflows, HITL resume).
  • Applies projection at logging completion / execution-core boundaries; avoids re-injecting raw values from streaming completion and HITL resume into already-sanitized logs.
  • Leaves handler inputs, retries, billing, and downstream block state on raw resolved values; adds extensive Vitest coverage across sanitizer, registry, executor, logging, and streaming paths.

Confidence Score: 5/5

No prior Greptile findings remain to re-score and no eligible follow-up inline comments were identified, so the PR appears safe to merge from this pass.

No blocking failure remains from outstanding prior threads or newly eligible incomplete-fix findings in this follow-up review.

Important Files Changed

Filename Overview
apps/sim/executor/utils/resolved-secret-trace-registry.ts New catalog/active-match registry with encrypted provenance, scope checks, and bounded export-for-value filtering for cross-boundary secret tracking.
apps/sim/lib/logs/execution/trace-secret-projection.ts Terminal projection that rewrites secret literals in trace trees (including large-value refs) without mutating runtime block state.
apps/sim/lib/logs/execution/logger.ts Completes logs with prepared/projected spans, preserves private provenance on PII-masked execution state, and adjusts size compaction of tool-call content.
apps/sim/executor/variables/resolvers/env.ts Records resolved env secrets into the trace registry only when the name exists on execution environment variables.
apps/sim/lib/workflows/executor/execution-core.ts Wires registry creation and trace projection into the main workflow execution completion path.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  EnvResolve["EnvResolver.recordResolved"] --> Registry["ResolvedSecretTraceRegistry"]
  Nested["MCP / function / mothership / nested WF"] --> Prov["Encrypted provenance import/export"]
  Prov --> Registry
  Runtime["Raw block I/O for execution"] --> Logs["buildTraceSpans / completeWorkflowExecution"]
  Registry --> Project["projectTraceSpansForSecrets"]
  Logs --> Project
  Project --> Persist["Persisted executionData.traceSpans"]
  Project --> Overview["Overview / export / UI logs"]
Loading

Reviews (5): Last reviewed commit: "secrets sanitization correctness" | Re-trigger Greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 95d4fd1. Configure here.

@BillLeoutsakosvl346
BillLeoutsakosvl346 marked this pull request as ready for review July 28, 2026 05:44
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

Comment thread apps/sim/lib/workflows/streaming/streaming.ts Outdated
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile please review the latest commit, including the workflow-overview sanitization follow-up and the streaming token-accounting correction.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e73afbe. Configure here.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

Demo:

Screen.Recording.2026-07-28.at.12.00.01.AM.mov

@icecrasher321 icecrasher321 changed the title Fix secret exposure in function and agent trace spans fix(sanitization): secret exposure in function and agent trace spans Jul 31, 2026
@gitguardian

gitguardian Bot commented Aug 1, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
35187658 Triggered Username Password 64f791f apps/desktop/src/main/browser-credentials/vault.test.ts View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

executionId,
workflowId,
data: { blockId, chunk },
data: { blockId, chunk, display },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SSE still sends raw stream text

High Severity

Workflow SSE stream:chunk and agent stream:thinking events still include the full raw chunk/text while projectLiveDisplayText may return only clearLiveDisplay or a sanitized display. Any SSE consumer (browser stream, reconnect, logging) can read resolved secrets even when the display copy is suppressed.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

body: JSON.stringify({ input, triggerType: 'manual', stream: true }),
body: JSON.stringify({
inputFromExecutionId: executionId,
triggerType: 'manual',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client ignores chunk display projection

Medium Severity

Log retry now sends only inputFromExecutionId, and the execute route sets input from stored workflowInput. Executions that predate persisted workflowInput load successfully but run with undefined input, unlike the removed client fallback that reconstructed input from starter block state.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

onBlockError: (data) => {
// Failures often skip stream:done — settle thinking/tool chrome here.
settleAgentStreamChrome(data.blockId, 'error')
agentStreamChrome.settleBlock(data.blockId, 'error')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream chunks ignore display projection

Medium Severity

Server stream:chunk events now include a sanitized display payload alongside raw chunk, but the workflow execution hook still buffers and forwards only data.chunk. That bypasses the new secret-safe streaming path while thinking deltas already honor display, so resolved secrets can still appear in chat streaming callbacks and in-memory chunk aggregation during server-side runs.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

@icecrasher321

Copy link
Copy Markdown
Collaborator

bugbot run

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

There are 5 total unresolved issues (including 3 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

return NextResponse.json(
{ ...payload, [RESOLVED_SECRET_PROVENANCE_FIELD]: provenance.exportProvenance() },
{ status: response.status, headers }
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MCP verify failure keeps success status

Medium Severity

When attachPrivateProvenance cannot read or parse the tool response (size limit, invalid JSON, etc.), it replaces the body with success: false but reuses the original HTTP status from the successful MCP call, so callers can see HTTP 200 alongside a failure payload.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

{ status: 404 }
)
}
input = sourceExecution.input

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retry omits legacy execution input

Medium Severity

Log retry now loads input only from persisted workflowInput via inputFromExecutionId. Executions that never stored that field still return found: true with missing input, so retry can start a run with undefined input instead of the previous starter/trigger payload.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 0acc905. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants